Biological image presentation device, biological image presentation method, program, and biological image presentation system

ABSTRACT

A biological image presentation device includes: an acquisition unit; a determination unit determining an image as a standard and an image for comparison; an extraction unit extracting, from the image as a standard, a position where a change of luminance value equal to or larger than a defined value is present; a detection unit detecting a position corresponding to the position extracted from the image for comparison; a division unit dividing the image as a standard on the basis of the position extracted; a mapping unit mapping the image for comparison to an area corresponding to each divided area of the image as a standard while modifying so as to conform to the shape of the divided area; and a display control unit switching and displaying an image for display in a display area by using the image as a standard and an image mapped by the mapping unit.

CROSS REFERENCES TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 12/731,942 filed on Mar. 25, 2010, whichapplication claims priority to Japanese Priority Patent Application JP2009-089413 filed in the Japan Patent Office on Apr. 1, 2009, the entirecontents of which is hereby incorporated by reference.

BACKGROUND

The present application relates to a biological image presentationdevice, a biological image presentation method, a program, and abiological image presentation system. The present application issuitable for application to the field of image processing.

A pathological sample is created by fixing a tissue slice sectioned froman excised tissue onto a glass slide and staining the tissue slice asoccasion demands, and stored. In general, if the pathological sample isstored for a long period of time, visibility on a microscope isdeteriorated due to deterioration or discoloring of the tissue slice.The pathological sample is used for diagnosis at institutions other thanthe institution, such as a hospital, which has created the pathologicalsample, and the pathological sample is generally delivered by mail.Accordingly, it takes a predetermined time to deliver the pathologicalsample.

Taking such a situation into consideration, a computer system issuggested in which an enlarged image of a tissue slice in a pathologicalsample is stored as image data, and the enlarged image is presented on adisplay screen (for example, see JP-T-2006-292999).

SUMMARY

A pathological sample is created for tissue slices at multiple positionsfrom the surface layer to the deep layer of the excised tissue, but acomplex operation should be made to find a specific cell of a tissueslice of any layer from a tissue slice of a layer different from therelevant layer by using the display screen. For this reason, there is aspecific demand for tissue slice images of respective tomographic layersfrom the surface layer to the deep layer to be three-dimensionallyviewed.

However, when the tissue slice images of the respective tomographiclayers from the surface layer to the deep layer are three-dimensionallyviewed, the tissue slices of the respective tomographic layers are notsubjected to processes, such as sectioning, staining, fixation, and thelike, so the tissue slices are not identical.

For this reason, when a tissue slice image of a different tomographiclayer is displayed from a tissue slice image of any tomographic layer,the relevant tissue slice image may be viewed as an entirely differentimage even though both images are identical.

Thus, it is desirable to provide a biological image presentation device,a biological image presentation method, a program, and a biologicalimage presentation system capable of improving visibility.

An embodiment provides a biological image presentation device. Thedevice includes an acquisition unit acquiring, at multiple positions onan axis as a standard of a biological region, images of sectionsorthogonal to the axis, a determination unit determining an image as astandard and an image for comparison with the image as a standard fromamong a plurality of images acquired by the acquisition unit, anextraction unit extracting, from the image as a standard, a positionwhere a change of luminance value equal to or larger than a definedvalue is present, a detection unit detecting a position corresponding tothe position extracted by the extraction unit from the image forcomparison, a division unit dividing the image as a standard on thebasis of the position extracted by the extraction unit, a mapping unitmapping the image for comparison to an area corresponding to eachdivided area of the image as a standard while modifying so as to conformto the shape of the divided area, and a display control unit switchingand displaying an image for display in a display area by using the imageas a standard and an image mapped by the mapping unit.

Another embodiment provides a biological image presentation method. Themethod includes the steps of acquiring, at multiple positions on an axisas a standard of a biological region, images of sections orthogonal tothe axis, determining an image as a standard and an image for comparisonwith the image as a standard from among a plurality of images acquiredin the step of acquiring the images, extracting, from the image as astandard, a position where a change of luminance value equal to orlarger than a defined value is present, detecting a positioncorresponding to the position extracted in the step of extracting theposition from the image for comparison, dividing the image as a standardon the basis of the position extracted in the step of extracting theposition, mapping the image for comparison to an area corresponding toeach divided area of the image as a standard while modifying so as toconform to the shape of the divided area, and switching and displayingan image for display in a display area by using the image as a standardand an image mapped in the step of mapping the image for comparison.

Still another embodiment provides a program. The program causes acomputer to execute the steps of acquiring, at multiple positions on anaxis as a standard of a biological region, images of sections orthogonalto the axis, determining an image as a standard and an image forcomparison with the image as a standard from among a plurality ofacquired images, extracting, from the image as a standard, a positionwhere a change of luminance value equal to or larger than a definedvalue is present, detecting a position corresponding to the extractedposition from the image for comparison, dividing the image as a standardon the basis of the extracted position, mapping the image for comparisonto an area corresponding to each divided area of the image as a standardwhile modifying so as to conform to the shape of the divided area, andswitching and displaying an image for display in a display area by usingthe image as a standard and a mapped image.

Yet another embodiment provides a biological image presentation system.The system includes a microscope and a data processor. The dataprocessor includes an acquisition unit acquiring, from the microscope,enlarged images of tissue slices sliced at multiple positions in a depthdirection of a tissue excised from a biological region, a determinationunit determining an image as a standard and an image for comparison withthe image as a standard from among a plurality of images acquired by theacquisition unit, an extraction unit extracting, from the image as astandard, a position where a change of luminance value equal to orlarger than a defined value is present, a detection unit detecting aposition corresponding to the position extracted by the extraction unitfrom the image for comparison, a division unit dividing the image as astandard on the basis of the position extracted by the extraction unit,a mapping unit mapping the image for comparison to an area correspondingto each divided area of the image as a standard while modifying so as toconform to the shape of the divided area, and a display control unitswitching and displaying an image for display in a display area by usingthe image as a standard and an image mapped by the mapping unit.

According to an embodiments, the comparison image is mapped to each areabased on a feature point while conforming to the area shape of thestandard image. For this reason, even though the images (tomographicimages) of the respective sections of a biological body correspond toeach other but are different in shape, the difference can be correctedwhile the shape of the internal organ itself can be maintained.Therefore, when an image for display in the display area is switched, itis possible to avoid the images before and after switching from beingviewed as entirely different images even though the images areidentical. As a result, a biological image presentation device, abiological image presentation method, a program, and a biological imagepresentation system capable of improving visibility can be realized.

Additional features and advantages are described herein, and will beapparent from the following Detailed Description and the figures.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a diagram schematically showing the configuration of abiological image presentation device.

FIG. 2 is a schematic view illustrating a group of glass slides ontowhich respective tissue slices in a depth direction are fixed.

FIG. 3 is a block diagram showing the configuration of a data processor.

FIG. 4 is a block diagram showing the functional configuration of a CPUwhich executes acquisition processing.

FIG. 5 is a schematic view illustrating acquisition of an image of eacharea of a tissue slice.

FIG. 6 is a block diagram showing the functional configuration of theCPU which executes image processing.

FIG. 7 is a photograph showing an extraction result of a feature point.

FIG. 8 is a photograph illustrating search of a corresponding point.

FIG. 9 is a photograph illustrating triangulation.

FIG. 10 is a photograph illustrating mapping of a triangular patch.

DETAILED DESCRIPTION

Embodiments of the present application will be described below in detailwith reference to the drawings. The description will be provided in thefollowing sequence.

1. Embodiment

1-1. Configuration of biological Image Presentation Device

1-2. Configuration of Data Processor

1-3. Acquisition Processing of Tissue Slice Image

1-4. Image Processing of Tissue Slice Image

1-5. Presentation Processing of Tissue Slice Image

1-6. Effects and the like

2. Other Embodiments

<1. Embodiment>

1-1. Configuration of Biological Image Presentation Device

FIG. 1 shows a biological image presentation system 1. The biologicalimage presentation system 1 includes a microscope 10 and a dataprocessor 20.

The microscope 10 has a surface (hereinafter, also referred to as aslide setting surface) on which a glass slide SG is set, and includes astage (hereinafter, also referred to as a movable stage) 11 which can bemoved parallel and orthogonally (xyz-axis direction) with respect to theslide setting surface.

The movable stage 11 is provided with a mechanism (hereinafter, alsoreferred to as a slide setting mechanism) 12 which sets a plurality ofglass slides SG contained in a glass slide container (not shown) at aset position on the slide setting surface one by one.

In this embodiment, a tissue slice is fixed onto the glass slide SG by apredetermined fixing method. As shown in FIG. 2, all or a part of tissueslices sectioned from a excised tissue of a biological body in a depth(deep portion) direction are fixed onto the respective glass slides, andthe glass slides are contained in the glass slide container in order,for example, from a glass slide onto which a surface layer-side (shallowsectioning position) tissue slice is fixed.

The tissue slice is stained as occasion demands. Staining includesgeneral staining represented by HE (hematoxylin-eosin) staining, Giemsastaining, or Papanicolaou staining, as well as fluorescent staining,such as FISH (Fluorescence In-Situ Hybridization) or enzyme antibodytechnique.

An optical system 13 is arranged on one surface of the movable stage 11,and an illumination lamp 14 is arranged on the other surface of themovable stage 11. Light of the illumination lamp 14 is reached from anopening formed at the set position on the slide setting surface asillumination light with respect to a biological sample SPL arranged onone surface of the movable stage 11.

The microscope 10 enlarges an image of a part of a tissue slice obtainedby illumination light at a predetermined magnification by a firstobjective lens 13A and a second objective lens 13B of the optical system13, and forms the image on an imaging surface of the imaging unit 15.

The microscope 10 also has an excitation light source 16 whichirradiates excitation light with respect to fluorescent stain. Whenexcitation light is irradiated from the excitation light source 16, themicroscope 10 causes excitation light to be reflected by a dichroicmirror 12C provided between the first objective lens 13A and the secondobjective lens 13B and to be directed to the first objective lens 13A.The microscope 10 focuses excitation light on the glass slide SGarranged on the movable stage 11 by the first objective lens 13A.

When the biological sample SPL fixed onto the glass slide SG issubjected to fluorescent stain, a fluorescent pigment emits light due toexcitation light. Light (hereinafter, also referred to as coloringlight) resulting from the light-emission transmits the dichroic mirror12C through the first objective lens 13A. Coloring light reaches thesecond objective lens 13B through an absorbent filter plate 13D providedbetween the dichroic mirror 12C and the second objective lens 13B.

The microscope 10 enlarges the image of coloring light by the firstobjective lens 13A, and absorbs light (hereinafter, also referred to asoutside light) other than coloring light by the absorbent filter plate13D. Then, the microscope 10 enlarges the image of coloring light withno outside light by the second objective lens 13B, and forms the imageon the imaging surface of the imaging unit 15.

The data processor 20 is configured to store the enlarged images(hereinafter, also referred to as enlarged slice images) of tissueslices (FIG. 2) of an excised tissue at multiple positions in the depth(deep portion) direction as a predetermined format of data.

Therefore, the biological image presentation system 1 can store thetissue slice for a long period of time without deteriorating the statesof fixing or staining, as compared with the glass slide SG itself isstored.

The data processor 20 is configured to construct a three-dimensionallycombined image (hereinafter, also referred to as a three-dimensionaltissue image) of the enlarged slice images, and to present thethree-dimensional tissue image or the enlarged slice images.

Therefore, the biological image presentation system 1 can present thestate of the excised tissue at various points of time, and as a result,the invasion depth (the degree of invasiveness) of a pathological changeand the like can be accurately determined.

1-2. Configuration of Data Processor

FIG. 3 shows the configuration of the data processor 20. The dataprocessor 20 has a configuration in which various kinds of hardware areconnected to a CPU (Central Processing Unit) 21, which performs overallcontrol of the biological image presentation system 1.

Specifically, a ROM (Read Only Memory) 22, a RAM (Random Access Memory)23 as a work memory of the CPU 21, a buffer memory 24, an operationinput unit 25 for inputting a command according to a user's operation,an interface 26, a display unit 27, and a storage unit 28 are connectedto the CPU 21 through a bus 29.

The ROM 22 stores programs for executing various kinds of processing.The movable stage 11, the slide setting mechanism 12, the illuminationlamp 14, the imaging unit 15, and the excitation light source 16 areconnected to the interface 26 through a predetermined communicationpath.

For the display unit 27, a liquid crystal display, an EL (ElectroLuminescence) display, or a plasma display is used. For the storage unit28, a magnetic disk represented by an HD (Hard Disk), a semiconductormemory, or an optical disk is used. A portable memory, such as a USB(Universal Serial Bus) memory or a CF (Compact Flash) memory, may beused.

The CPU 21 loads a program corresponding to a command from the operationinput unit 25 from among a plurality of programs stored in the ROM 22 onthe RAM 23, and appropriately controls the buffer memory 24, the displayunit 27, and the storage unit 28 in accordance with the loaded program.

The CPU 21 is also configured to appropriately control the movable stage11, the slide setting mechanism 12, the illumination lamp 14, theimaging unit 15, and the excitation light source 16 through theinterface 26 in accordance with the loaded program.

1-3. Acquisition Processing of Tissue Slice Image

When an acquisition command of an enlarged slice image is received fromthe operation input unit 25, the CPU 21 functions as a slide settingmechanism 41, an enlarged slice image acquisition unit 42, and a datastorage unit 43 in accordance with a program corresponding to theacquisition command, as shown in FIG. 4.

The slide setting mechanism 41 controls the slide setting mechanism 12to set a plurality of glass slides SG contained in the glass slidecontainer one by one at the set position on the slide setting surface.

Each time a glass slide SG is set by the slide setting mechanism 41, theenlarged slice image acquisition unit 42 acquires an enlarged sliceimage of a tissue slice fixed onto the set glass slide SG.

Specifically, as a first step, the enlarged slice image acquisition unit42 adjusts the focus position of the optical system 13 and also thesensitivity of the imaging unit 15.

With regard to adjustment of the focus position, for example, a methodis used in which the movable stage 11 is controlled on the basis of apixel value represented by captured data output from the imaging unit15, such that the relative positions of the optical system 13 and theimaging surface of the imaging unit 15 in the optical axis direction arevaried.

With regard to adjustment of the sensitivity of the imaging unit 15, forexample, a method is used in which all or part of the irradiation amount(light intensity) of excitation light of the illumination lamp 14 or theexcitation light source 16, the exposure time in the imaging unit 15,and the aperture ratio of the aperture stop are varied.

As a second step, for example, as shown in FIG. 5, the enlarged sliceimage acquisition unit 42 moves the movable stage 11 along the slidesetting surface such that a tissue slice ETS (enlarged slice image)enlarged through the objected lenses 13A and 13B is assigned to animaging range AR. In FIG. 5, the area of the tissue slice ETS which isassigned to the imaging range AR does not overlap other areas, but apart of an adjacent area may overlap the area of the tissue slice ETS.

As a third step, the enlarged slice image acquisition unit 42 connectsthe area by using captured data of the area assigned to the imagingrange AR so as to generate an enlarged slice image.

As a fourth step, the enlarged slice image acquisition unit 42 generatesan enlarged slice image as an image of different resolution for eachlevel, and divides each level of enlarged slice image in terms of apredetermined block (hereinafter, also referred to as a tile).

In this way, the enlarged slice image acquisition unit 42 is configuredto generate the enlarged slice image of the tissue slice fixed onto theglass slide SG to correspond each level of resolution which should bedisplayed in the display area.

When the enlarged slice image corresponding to each level of resolutionis generated, the data storage unit 43 generates a reduced image(hereinafter, also referred to as a thumbnail slice image) of theenlarged slice image corresponding to the defined level of resolutionand identification information regarding the enlarged slice image.

The data storage unit 43 is configured to record the enlarged sliceimage corresponding to each level of resolution and the thumbnail sliceimage in the storage unit 28 as a predetermined format of dataassociated with identification information.

The identification information is information which includes, forexample, the slide number, the name, assigned number, sex, and age of aperson (patient) whose tissue slice is collected, the collection date,and the like. The slide number is given, for example, in order from aglass slide, onto which a tissue slice (a tissue slice which issectioned earlier) as a shallow sectioned surface of an excised tissueis fixed, in terms of tissue slice-collected persons.

As a method of acquiring identification information, for example, amethod is used in which identification information is acquired from abarcode attached to a glass slide SG set at the set position, oridentification information is input from the operation input unit 25 ata predetermined timing. The method of acquiring identificationinformation is not limited to the illustration.

In this way, when the acquisition command of the enlarged slice image isgiven, the CPU 21 is configured to store the enlarged images of a tissueslice (FIG. 2) on a plurality of slice surfaces in the depth (deepportion) direction as data.

1-4. Image Processing of Tissue Slice Image

When the above-described acquisition processing ends, the CPU 21functions as a three-dimensional image generation unit 51 and adifferent resolution image generation unit 52 in accordance with aprogram regarding image processing, as shown in FIG. 6.

The three-dimensional image generation unit 51 generates athree-dimensional image from the enlarged slice image on the respectiveslice surfaces for each tissue slice-collected person, and isfunctionally divided into an image determination unit 61, a templatematching unit 62, a feature point extraction unit 63, a correspondingpoint detection unit 64, a division unit 65, and a texture mapping unit66.

The image determination unit 61 determines an enlarged slice image as acriterion (hereinafter, also referred to as a criterion tomographicimage) and an enlarged slice image for comparison with the criteriontomographic image (hereinafter, also referred to as a referencetomographic image) from a plurality of enlarged slice images.

In an embodiment, the respective enlarged slice images are determined asa criterion tomographic image in order from an enlarged slice imagewhose sectioning position is shallow, and an enlarged slice image whichis located on the surface layer side or deep layer side with respect tothe sectioning position of the criterion tomographic image is determinedas a reference tomographic image.

The template matching unit 62 positions the reference tomographic layerwith respect to the criterion tomographic layer. For the positioningmethod, for example, a method is used in which the reference tomographicimage is moved parallel or rotated such that the correlation with thepixel level at the criterion tomographic image is minimized.

The feature point extraction unit 63 extracts a position where a changeof luminance value stands out (a position where a change of luminancevalue equal to or smaller than a defined value is present) as a featurepoint from the criterion tomographic image by using an extractionalgorithm called Harris corner.

FIG. 7 shows a photograph in which feature points extracted from anenlarged slice image are attached in the form of black points. As willbe apparent from FIG. 7, the extraction algorithm is advantageous inthat no feature point is extracted from a portion where gradation ispresent. The extraction algorithm is also advantageous in that it isrobust with respect to rotation of an image to be extracted.

The corresponding point detection unit 64 detects a point (hereinafter,referred to as a corresponding point) in the criterion tomographic imagecorresponding to the feature point from the reference tomographic image.

Specifically, as a first step, the corresponding point detection unit 64sequentially determines each feature point in the criterion tomographicimage as a point that is an attention target (hereinafter, referred toas an attention point).

As a second step, as shown in FIG. 8, the corresponding point detectionunit 64 recognizes a block of a predetermined size (hereinafter,referred to as an attention block) ABL around the determined attentionpoint AP.

The corresponding point detection unit 64 searches a block havinghighest similarity to the attention block ABL from a search range AR ofa size larger than the attention block ABL on the basis of a position ofthe reference tomographic image corresponding to the attention point AP,for example, by a normalized correlation method.

As a third step, when the highest similarity (correlation value) of ablock DBL with respect to the attention block ABL is equal to or largerthan a set value, the corresponding point detection unit 64 detects thecenter of the relevant block DBL as a corresponding point XP of theattention point AP.

Meanwhile, when the highest similarity (correlation value) of theattention block ABL and the block DBL is smaller than the set value, thecorresponding point detection unit 64 is configured to determine thatthere is no corresponding point XP of the attention point AP and todelete the attention point AP.

Therefore, even when a portion which does not have a feature as a tissueform is extracted as a feature point by the feature point extractionunit 63 since the tissue slice is damaged, expanded/reduced, or airbubbles are generated during sectioning, staining, and fixation, therelevant portion is deleted by the corresponding point detection unit64.

As described above, the three-dimensional image generation unit 51 isconfigured to accurately extract, as a feature point, a portion whichtruly has a feature as a tissue form.

When the corresponding point in the reference tomographic image isdetected, the division unit 65 divides the criterion tomographic imageinto triangular areas with a feature point as an apex (hereinafter, alsoreferred to as triangular patches) by using a division algorithm calledDelaunay triangulation.

FIG. 9 shows a photograph in which division boundaries dividing thecriterion tomographic image are attached as black lines. As will beapparent from FIG. 9, as the number of feature points, a complex tissueform is present, and the relevant form is divided finer.

The texture mapping unit 66 recognizes an area corresponding to eachtriangular patch of the criterion tomographic image on the basis of thecorresponding point in the reference tomographic image. If the tissueslice is damaged, expanded/reduced, or air bubbles are generated, eacharea does not conform to the shape of the corresponding triangular patchsince the absolute positions of the corresponding point and the featurepoint of the criterion tomographic image differ.

As shown in FIG. 10, the texture mapping unit 66 maps each pixel of thereference tomographic image to a corresponding position of athree-dimensional object while modifying each the area so as to conformto the shape of the corresponding triangular patch.

Therefore, even though the areas in the respective tomographic imagescorrespond to each other but are different in shape, the difference iscorrected while the shape of the internal organ itself is maintained. Inaddition, the gap between the sectioning positions of the excised tissuein the deep layer direction differ depending on trial cuts or skills,and the difference is reflected in the shapes of the areas of therespective tomographic images even though they correspond to each other,but this difference is also corrected while the shape of the internalorgan itself is maintained.

As a complex tissue form is present, the correction unit (triangularpatch) becomes finer, so the shape of the internal organ between thetomographic layers is accurately maintained regardless of complexity ofthe tissue form.

In this way, the three-dimensional image generation unit 51 isconfigured to map the enlarge slice images on the respective slicesurfaces for each tissue slice-collected person, and to store the mappedenlarged slice images (hereinafter, also referred to as athree-dimensional slice image) in the storage unit 28 as a predeterminedformat of data associated with identification information.

For comparison, the mapped enlarged slice images include an enlargedslice image (criterion tomographic image) whose sectioning position isshallowest.

When the three-dimensional slice image is generated from the enlargedslice images, the different resolution image generation unit 52generates the three-dimensional slice image as an image of differentresolution for each level, and divides each level of three-dimensionalslice image in terms of tiles.

The different resolution image generation unit 52 is configured to storethe three-dimensional image corresponding to each level of resolution inthe storage unit 28 as a predetermined format of data associated withidentification information.

1-5. Presentation Processing of Tissue Slice Image

When a presentation command of a tissue image is given from theoperation input unit 25, the CPU 21 displays a screen (hereinafter, alsoreferred to as a thumbnail image), which shows the thumbnail sliceimages to be stored in terms of tissue slice-collected persons, on thedisplay unit 27 in accordance with a program corresponding to thepresentation command.

When a thumbnail slice image which should be displayed as an enlargedslice image is selected, the CPU 21 displays the thumbnail slice imageand a pointer capable of moving the thumbnail slice image in accordancewith the operation of the operation input unit 25.

In this case, the CPU 21 selects an enlarged slice image correspondingto an initial resolution from among the enlarged slice imagescorresponding to the respective levels of resolution associated with thethumbnail slice image for selection. The CPU 21 reads a tile in aportion (hereinafter, also referred to as an image for display)corresponding to the display area from the selected enlarged slice imageand displays the tile.

When a position for display is designated from the operation input unit25 (when the position of the pointer is changed), the CPU 21 reads atile in the image for display on the basis of the designated positionand displays the tile.

When resolution for display is designated, the CPU 21 reads a tile inthe image for display from an enlarged slice image corresponding to thedesignated resolution on the basis of the position of the pointer at thetime of designation and displays the tile.

In this way, the CPU 21 is configured to switch the position in thesurface direction or resolution of the enlarged slice image for displayin the display area in accordance with designation from the operationinput unit 25.

The thumbnail slice image for selection, the pointer, and the image fordisplay are displayed as the same display layer by switching thethumbnail screen or in a layer higher than the display layer of thethumbnail screen.

Meanwhile, when a thumbnail slice image for display as athree-dimensional tissue image is selected, the CPU 21 displays athree-dimensional object and a pointer capable of moving thethree-dimensional object in accordance with an operation of theoperation input unit 25.

In this case, the CPU 21 selects a three-dimensional slice imagecorresponding to the initial resolution and an initial tomographicsurface from among three-dimensional slice images corresponding to therespective levels of resolution associated with the thumbnail sliceimage for selection. The CPU 21 reads, from the storage unit 28, a tilein a portion (hereinafter, also referred to as an image for display)corresponding to the display area from the selected three-dimensionalslice image, and displays the tile in the display area.

When a position for display is designated from the operation input unit25 (when the position of the pointer is changed), the CPU 21 reads, fromthe storage unit 28, a tile in the image for display from athree-dimensional slice image corresponding to a tomographic surface atthe designated position on the basis of the designated position. Then,the CPU 21 displays the tile read from the storage unit 28 in thedisplay area.

When resolution for display is designated, the CPU 21 reads a tile inthe image for display from a three-dimensional slice image correspondingto the tomographic surface at the time of designation from among thethree-dimensional slice images corresponding to the designatedresolution on the basis of the position of the pointer at the time ofdesignation, and displays the tile.

As described above, the CPU 21 is configured to switch the position inthe depth direction or the surface direction or resolution of athree-dimensional slice image for display in the display area inaccordance with designation from the operation input unit 25.

The three-dimensional object, the pointer, and the image for display aredisplayed as the same display layer by switching the thumbnail screen orin a layer higher than the display layer of the thumbnail screen.

In this embodiment, when the image for display in the display area isdisplayed in the display area, the CPU 21 predicts the next image fordisplay (hereinafter, also referred to as a display predicted image) inaccordance with the current image for display. The CPU 21 reads a tilein the display predicted image from the storage unit 28, and stores thetile in the buffer memory 24.

Then, the CPU 21 reads a tile of the same portion as the displaypredicted image in the next image for display in the display area fromthe buffer memory 24, reads a tile of a portion different from thedisplay predicted image from the storage unit 28, and displays the tilesin the display area.

Thus, the CPU 21 is configured to smoothly display adjacent images fordisplay in the depth direction or the tomographic surface direction.

1-6. Effects and the Like

With the above-described configuration, when the enlarged slice imagesof the tissue slices sliced at multiple positions in the depth directionof the excised tissue are acquired, the biological image presentationsystem 1 maps the enlarged slice images while modifying the enlargedslice images in terms of triangular patched (see FIG. 10).

Then, the biological image presentation system 1 switches the positionin the depth direction on the three-dimensional image for display in thedisplay area in accordance with designation from the operation inputunit 25.

Therefore, even when the areas in the respective tomographic imagescorrespond to each other but the shape or the gap between the sectioningpositions in the depth direction with respect to the excised tissuediffers, the biological image presentation system 1 can correct thedifference while maintaining the shape of the internal organ itself.

This difference is caused due to many artificial steps, such assectioning, staining, fixation, and the like, during the pathologicalsample is created. Therefore, the enlarged slice image is mapped whilebeing modified in terms of triangular patches, and it is particularlyuseful for the pathological sample.

In the case of a pathological sample in which a fluorescent mark islinked to a specific gene of a tissue (FISH: Fluorescence In-SituHybridization), even when the fluorescent mark overlaps in the deeplayer direction, the biological image presentation system 1 canaccurately detect an increase/decrease in the number of genes.

2. Other Embodiments

In an embodiment, the enlarged images of the tissue slices sliced atmultiple positions in the depth direction of the excised tissue of thebiological body are acquired. However, the tomographic images to beacquired are not limited thereto.

For example, at multiple positions on a body axis, such as a head, achest, or an abdomen, CT (Computed Tomography) images, MRI (MagneticResonance Imaging) images, PET (Positron Emission Tomography) images, orultrasonic images of sections orthogonal to the body axis may beacquired. What is important is that, at multiple positions on an axis asa criterion in a biological region, images (tomographic images) ofsections orthogonal to the axis are acquired.

With regard to acquisition of the tomographic image, the enlarged sliceimage acquisition unit 42 is used which acquires the tomographic imagefrom the microscope 10 in real time. However, the acquisition unit ofthe tomographic image is not limited thereto.

For example, an acquisition unit may be used which, at multiplepositions on an axis as a criterion, acquires images (tomographicimages) of sections orthogonal to the axis at different magnifications.With this acquisition unit, the state of the excised tissue can bepresented at more points of view, so it is useful in that the invasiondepth (the degree of invasiveness) of a pathological change and the likecan be accurately determined.

In addition, an acquisition unit may be used which acquires images froma data storage medium, in which data representing the images(tomographic images) of sections orthogonal to the axis at multiplepositions on an axis as a criterion is stored. This acquisition unit isparticularly useful, for example, when the location where thetomographic image is acquired is away from a diagnosis location.Furthermore, an acquisition unit may be used in which the tomographicimages, such as enlarged slice images, CT images, PET images, orultrasonic images, are selectively acquired from a corresponding device(a microscope, CT device, a PET device, or an ultrasonic device) and adata storage medium.

As the data storage medium, for example, a package medium, such as aflexible disk, a CD-ROM (Compact Disk-Read Only Memory), or a DVD(Digital Versatile Disc), or a semiconductor memory or a magnetic diskin which data is temporarily or permanently stored may be used. Withregard to the method of acquiring data from the data storage medium, awired or wireless communication medium, such as a local area network orInternet, or digital satellite broadcasting, may be used.

In the above-described embodiment, the respective enlarged slice imagesare determined as the criterion tomographic image in order from theenlarged slice image whose sectioning position is shallow. However, theenlarged slice image of the tissue slice of the shallowest sectionedsurface or the like may be fixed as the criterion tomographic image. Inthis case, the remaining enlarged slice images are determined as thereference tomographic image.

The criterion tomographic image is not limited to a single layer. Forexample, the enlarged slice image of the tissue slice of the shallowestsectioned surface and the enlarged slice image of the tissue slice ofthe deepest sectioned surface may be used as the criterion tomographicimage. In this case, a notification that a criterion tomographic imageshould be designated may be made, and an enlarged slice image selectedby the operation input unit 25 may be determined as the criteriontomographic image.

When the respective enlarged slice images are determined as thecriterion tomographic image in order from the enlarge slice image whosesectioning position is shallow (or deep), the reference tomographicimage is an enlarged slice image which is located adjacent to thesurface layer side or deep layer side with respect to the sectioningposition of the criterion tomographic image. In this case, a pluralityof reference tomographic images may be set. When this happens, a moreproper feature point can be obtained.

The method of determining the criterion tomographic image and thereference tomographic image is not limited to the illustration, and maybe modified in various forms.

In an embodiment, the processing in the three-dimensional imagegeneration unit 51 and the different resolution image generation unit 52is performed before the presentation command of the tissue slice imageis given. However, the time at which the processing is performed is notlimited to the embodiment. For example, the processing may be performedwhen an image for display in the display area is read from the storageunit 28. Part of the processing in the three-dimensional imagegeneration unit 51 and the different resolution image generation unit 52may be performed before the presentation command of the tissue sliceimage is given, and the remaining processing may be performed after thepresentation command is given.

In an embodiment, the image for display in the display area is switchedin accordance with designation (the position of the pointer) of theoperation input unit 25. However, the switching method is not limited tothe embodiment.

For example, a switching method may be used in which the image fordisplay in the display area is switched in a predetermined order, suchas a raster scan order, from the outermost layer or the deepest layer.With the switching method, if switching of the image for display stopswhen a stop command is received from the operation input unit 25, anoperation with respect to a viewer can be simplified, as compared withthe above-described embodiment.

In an embodiment, when a feature point is extracted from an enlargedslice image, the extraction algorithm called Harris corner is used.However, the extraction algorithm is not limited thereto. For example,an extraction algorithm, such as Moravec, vector tracer, SUSAN (SmallestUnivalue Segment Assymilating Nucleus), or SIFT (Scale Invariant FeatureTransformation), may be used. Of course, other extraction algorithms maybe used. Any algorithm may be used insofar as a position where a changeof luminance value stands out (a position where a change equal to orsmaller than a defined value is present) can be extracted.

In an embodiment, when an enlarged slice image is divided, the divisionalgorithm calls Delaunay triangulation is used. However, the divisionalgorithm is not limited thereto. For example, a division algorithm,such as Voronoi, may be used. Of course, other division algorithms maybe used. Any algorithm may be used insofar as it divides an image into aplurality of areas on the basis of a feature point.

Although in an embodiment, the single buffer memory 24 is connected tothe bus 29, a plurality of buffer memories 24 may be connected. Whenthis happens, the image for display may be smoothly shifted regardlessof the amount of image data for display or the display movement speed.

In an embodiment, the three-dimensional image is stored in the storageunit 28 of the data processor 20 through the bus 29. The storage unit 28may be provided outside the data processor 20, instead of being providedin the data processor 20. A communication medium of data with respect tothe storage unit 28 is not limited to the bus 29, and for example, awired or wireless communication medium, such as a local area network orInternet, or digital satellite broadcasting, may be used.

The present application may be used for biotechnology-based industries,such as experiments of biology, development of medicine, and follow-upof patients according to an embodiment.

It should be understood that various changes and modifications to thepresently preferred embodiments described herein will be apparent tothose skilled in the art. Such changes and modifications can be madewithout departing from the spirit and scope of the present subjectmatter and without diminishing its intended advantages. It is thereforeintended that such changes and modifications be covered by the appendedclaims.

The invention is claimed as follows:
 1. A biological image presentationdevice comprising: an acquisition unit acquiring, at multiple positionson an axis as a standard of a biological region, images of sectionsorthogonal to the axis; a determination unit determining an image as astandard and an image for comparison with the image as a standard fromamong a plurality of images acquired by the acquisition unit; anextraction unit extracting, from the image as a standard, a positionwhere a change of luminance value equal to or larger than a definedvalue is present; a detection unit detecting a position corresponding tothe position extracted by the extraction unit from the image forcomparison; a division unit dividing the image as a standard on thebasis of the position extracted by the extraction unit; a mapping unitmapping the image for comparison to an area corresponding to eachdivided area of the image as a standard while modifying so as to conformto the shape of the divided area; and a display control unit switchingand displaying an image for display in a display area by using the imageas a standard and an image mapped by the mapping unit.
 2. The deviceaccording to claim 1, wherein the acquisition unit acquires enlargedimages of tissue slices sliced at multiple positions in the depthdirection of a tissue excised from the biological region.
 3. The deviceaccording to claim 2, wherein the detection unit searches, for eachposition extracted by the extraction unit, a block having maximumsimilarity with respect to a search block on the basis of the relevantposition, when the similarity is equal to or larger than a set value,detects a predetermined position of the block as the correspondingposition, and when the similarity is smaller than the set value, deletesthe position.
 4. The device according to claim 2, further comprising: adisplay unit division unit dividing the image as a standard and theimage mapped by the mapping unit into display unit blocks, wherein, whenthe image for display in the display image is displayed, the displaycontrol unit predicts the next image for display in accordance with thecurrent image, stores the display unit blocks of the predicted image ina buffer memory, reads the display unit blocks of the same portions ofthe next image for display in the display area as the predicted image,from the buffer memory, and reads the display unit blocks of portionsdifferent from the predicted image from a storage unit.
 5. The deviceaccording to claim 4, wherein the display unit division unit generatesthe image as a standard and the image mapped by the mapping unit asimages corresponding to a plurality of levels of resolution, and dividesthe image corresponding to each level of resolution into the displayunit block.
 6. A biological image presentation method comprising:acquiring, at multiple positions on an axis as a standard of abiological region, images of sections orthogonal to the axis;determining an image as a standard and an image for comparison with theimage as a standard from among a plurality of images acquired in thestep of acquiring the images; extracting, from the image as a standard,a position where a change of luminance value equal to or larger than adefined value is present; detecting a position corresponding to theposition extracted in the step of extracting the position from the imagefor comparison; dividing the image as a standard on the basis of theposition extracted in the step of extracting the position; mapping theimage for comparison to an area corresponding to each divided area ofthe image as a standard while modifying so as to conform to the shape ofthe divided area; and switching and displaying an image for display in adisplay area by using the image as a standard and an image mapped in thestep of mapping the image for comparison.
 7. A program which causes acomputer to execute the steps of: acquiring, at multiple positions on anaxis as a standard of a biological region, images of sections orthogonalto the axis; determining an image as a standard and an image forcomparison with the image as a standard from among a plurality ofacquired images; extracting, from the image as a standard, a positionwhere a change of luminance value equal to or larger than a definedvalue is present; detecting a position corresponding to the extractedposition from the image for comparison; dividing the image as a standardon the basis of the extracted position; mapping the image for comparisonto an area corresponding to each divided area of the image as a standardwhile modifying so as to conform to the shape of the divided area;switching and displaying an image for display in a display area by usingthe image as a standard and a mapped image.
 8. A biological imagepresentation system comprising: a microscope; and a data processor,wherein the data processor includes an acquisition unit acquiring, fromthe microscope, enlarged images of tissue slices sliced at multiplepositions in the depth direction of a tissue excised from a biologicalregion, a determination unit determining an image as a standard and animage for comparison with the image as a standard from among a pluralityof images acquired by the acquisition unit, an extraction unitextracting, from the image as a standard, a position where a change ofluminance value equal to or larger than a defined value is present, adetection unit detecting a position corresponding to the positionextracted by the extraction unit from the image for comparison, adivision unit dividing the image as a standard on the basis of theposition extracted by the extraction unit, a mapping unit mapping theimage for comparison to an area corresponding to each divided area ofthe image as a standard while modifying so as to conform to the shape ofthe divided area, and a display control unit switching and displaying animage for display in a display area by using the image as a standard andan image mapped by the mapping unit.